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Abstract — A major objective In this paper is the application of 
connectionist architectures for fast and robust on-line learning 
of dynamic relations used in robot control at the executive 
hierarchical level. The proposed connectionist robot controllers 
as new feature use decomposition of robot dynamics in the space 
of internal robot coordinates. In this way, this method enables 
the training of neural networks on the simpler input-output 
relations with significant reduction of learning time. The proposed 
controller structure comprises a form of intelligent feedforward 
control in the frame of decentralized control algorithm with 
feedback-error or driving torque error learning method. The 
another important features of these new algorithms are fast and 
robust convergence properties because the problem of adjusting 
the weights of Internal hidden units is considered as a problem 
of estimating parameters by recursive least square method and 
Kalman Filter approach. From simulation examples of robot 
trajectory tracking it is shown that when a sufficiently trained 
network Is desired, the learning speed of the proposed algorithms 
Is faster than that of the standard back propagation algorithm. 

I. Introduction 

THE powerful development of flexible manufacturing sys- 
tems with high and complex demands for all components 
of production process, leads toward design of intelligent 
manipulation robots [1J, [2]. We can define these robots 
as autonomous machines capable of learning, making deci- 
sions, exercising fault tolerance, being robust and adaptive 
to internal and external disturbances, compensating for un- 
certainties, diagnosing failures, identifying failed components, 
and recovering from errors. Such robots also, can perform 
anthropomorphic tasks in an unfamiliar or familiar working 
environment. There has been a significant effort in making 
robot more intelligent by integrating advanced sensor sys- 
tems as vision, tactile sensing, etc. But, one of the major 
and ultimate steps in robotic research is the formulation 
and development of intelligent control algorithms which can 
further improve the performance of robotic systems, using 
control strategies generated by human intelligent functions as 
perception, association, reasoning, generalization or learning. 

The basic problems for the conventional model-based 
control of manipulation robots involve the state variables- 
dependency of the robot dynamic model, the expressive 
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coupling between robot subsystems, coping with structured 
and unstructured uncertainties and time-dependency of robot 
parameters. As is well known, none of the conventional 
robot controllers can not provide desirable solution for these 
problems, because traditional control laws generally are based 
on a model that provides incomplete information and only 
partially known or inaccurately defined parameters. Also, 
the conventional algorithms are extremely sensitive to the 
lack of sensor information and to unplanned events and 
unfamiliar situations in the robot working environment. Robot 
performance is not able to capture and use past experience and 
available human expertise. All previously mentioned facts and 
examples provide a motivation for robotic intelligent control 
and emphasize the necessity that efficient robotic intelligent 
control must be based on the learning, generalization and 
self-organizing capabilities. 

The conventional adaptive and non-adaptive control algo- 
rithms comprise a robot control problem during execution 
of single robot trajectories without considering of repetitive 
motion. Hence, in terms of learning, almost all manipulation 
robots are memoryless. In this way, the previously acquired 
experience about dynamic robot model and control algorithms 
is not applied in robot control synthesis. Using a training 
process that involves repeating a control task and recording the 
results accumulated in the entire processii, should thus steadily 
improve performance. Also, state variables-dependency of 
robot dynamics may be solved by learning and storing solution, 
while time-dependency of robot parameters requires an on-line 
learning approach. If by means of a learning control algorithm, 
a movement was correctly realized, quite different and faster 
movement could be controlled using generalization properties 
of learning algorithm. Hence, one of the primary goals in 
intelligent control of manipulation robots is the addition of 
learning and generalization capabilities to the conventional 
non-adaptive and adaptive control algorithms. 

The recent research reports and extensive simulation stud- 
ies carried out on models containing neural networks have 
demonstrated an ability to identify and control a sophisticated 
manipulation robots [3]— [15]. From a systems theoretic point 
of view, we can say that multilayer neural networks used in 
robot control in most cases represent static nonlinear mappings 
as a special part of pattern recognition problem. In this case 
the patterns to be recognized are the signals of "change'* that 
map in "control action" signals in the aim of desired control 
goals. The neural network controller should recognize and 
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isolate signals of "change" in real-lime conditions, and using 
learning by experience and generalization properties, to control 
efficiently system behavior. Through the training process, the 
model uncertainties are eliminated, and thus, neural networks 
serve as a compensation tools in control systems. 

In this paper, our purpose is presentation of new robot con- 
trol learning algorithms with fast and robust learning properties 
using special connectionist (neural network) architectures. The 
major concern is the application of neural networks in robot 
control at executive hierarchical level (motion control prob- 
lem) for learning inverse dynamic model of robot mechanism 
in the case where exact robot dynamics is generally unknown. 

The several neural network models and learning schemes 
were applied to learning of robot dynamics, recently. The 
one of main distinction between these methods is in extent 
of the knowledge about dynamic models which is used in 
design procedures. In this context, the term "knowledge" is 
related for the type and structure of the analytical form of a 
robot dynamic model. Some methods use in design procedure 
complete available information about robot model [10]-[12], 
The fundamental approach from this group is, for example, 
that of Kawato [10], [12], who uses a neural structure based 
on a nonrecurrent single-layer feedforward neural network. 
This approach has a deterministic nature, but there are several 
drawbacks related primarly to the inherent complexity of the 
implementation of a complete model of robot dynamics and 
poor generalization properties. At the other side of dynamic 
connectionist approaches are methods that use "black-box" 
approach in design of neural network algorithms for robot 
dynamic control [3], [5j, [8], [9], [14], [15]. In this case, neural 
networks can be used as very general computation models. 
Although a pure neural network approach without knowledge 
about robot dynamics may be promising, it is important to 
notice that this approach will not be very practical due high 
dimensionality of input-output spaces. In this way the training 
by pure connectionist models would require a neural network 
of impractical size and unreasonable number of repetition 
cycles. 

Hence, proposed connectionist controller in this paper as 
new feature uses principle of decomposition of robot dynamics 
in the space of internal robot coordinates. The results are sim- 
plified dynamic mappings and learning feasibility of robotic 
dynamics for larger systems with a significant reduction of 
learning time. This method includes a priori knowledge about 
robot dynamics, which instead of being particular knowledge 
corresponding to a certain class of models, incorporates gen- 
eral knowledge of robot dynamics. 

The other important feature of this new control structure is 
fast convergence properties, because the problem of adjusting 
the weights of internal hidden units is considered as a problem 
of estimating parameters by recursive least square method 
(RLS) in deterministic case and Kalman filter (KF) approach in 
stochastic case. In this way, special new algorithms with time- 
varying learning rate can yield benefits for learning speed and 
generalization. Also in this paper, adaptive gain of activation 
functions [16] in learning algorithm is included. It is shown 
that new learning rules with respect to adaptive gain gready 
increases learning speed. 



The final result of approach proposed in this paper is 
a trainable robot controller architecture that uses a neural 
network model as a form of intelligent feedforward control 
in the frame of decentralized control algorithm with training 
by a feedback-error learning method . 

n. Robot Control Problem at Executive 
Hierarchical Level and Connectionist Solution 

In contemporary robotic systems, there is a need for more 
flexible and robust robot controllers in order to take full advan- 
tage of the inherent flexibility and versatility of manipulation 
robots. Difficulties in the solution of control problem arise in 
various ways. 

The one of the most important problems is high nonlinearity 
of robot dynamics model with expressive couplings between 
robot subsystems. Based on well-known equations of rigid 
body mechanics, a dynamic model of manipulation robot in 
the absence of friction and other disturbances (deterministic 
model) can be written as 

P = /(</, </, q t 8) = H(g t 8)q + h( gi g y 6) (1 ) 

or 

P = /(?, </, <7, 8) = H(Q> *)$ + fC(q, B)q + g(q, 6) (2) 

where P € IC — is the vector of driving torques or forces; 
H(q t 9) : R n x8 R nxn is the inertial matrix of the system; 
h(q t q,8) : R n x R n x 8 => R n is the vector which includes 
Centrifugal, Coriolis and gravitational effects; C(q, 6) : iT* x0 
=> R n xR n xR n is the matrix which includes Centrifugal and 
Coriolis effects; g(q,0) : R n x 9 R n is the gravitational 
vector; 8 € ii"' is the system parameter vector; n— is the 
number of degree of freedom; nt — is the number of system 
parameters. 

A common and conventional way for robot control repre- 
sents local PED regulators for each degree of freedom of the 
robotic mechanism |17J 

u = u /b = -KP e-KDi- KI J edt (3) 

where utR n is the control input; u^e/P* is the feedback 
control; KPeR nXn — is the matrix of local position feedback 
gains; KDtK l * n — is the matrix of local velocity feedback 
gains; KItR nXn — is the matrix of local integral feedback 
gains; e = q — qj — is the feedback error (eeiT 1 ); q and qa 
are the real and nominal internal coordinates (qeR n , ^ci? n ). 

However, this control law is not adequate for advanced 
industrial robots that must demonstrate a capability for high 
precision and speed in a complex working environment. The 
influence of couplings between the subsystems is substantial, 
and we have to include as a solution "dynamic" control [17], 
which takes the dynamic model of robot mechanism in control 
synthesis as a form of feedforward control. On the basis of the 
above, we can apply a decentralized control algorithm [17]: 

u = u ff ~KPe-KDc-K1 J edt (4) 

where Uf/eRJ 1 is nominal centralized or decentralized feedfor- 
ward control which is off-line synthesized using the integral 
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robot model (model of mechanism with the model of robot 
actuators). 

Also, as another solution we can apply computed torque 
algorithm for the calculation of command driving torque f 171: 

P = H{q, 6) [q d + KP (q -q d )+KD(q- q d )] + h(q t q, 9) (5) 

where #(<j, 0) and k(g,g,0) are the estimates of H{q i 9) and 
h[q,q,B) % respectively. 

However, in controller design procedure, we have to cope 
with structured uncertainties (inaccuracies of model param- 
eters and additive disturbances), unstructured uncertainties 
(umodelled high frequency dynamics as structural resonant 
modes, neglected time-delays, actuator dynamics, sampling 
effects, etc.) and measurement noise. Also, lime- varying nature 
of robot parameters and variability of robot tasks represent 
additional difficulties for the control system. In this case, the 
conventional non-adaptive algorithms are not robust enough, 
because these algorithms compensate only a small number of 
the uncertainties noted. Hence, a more suitable approach would 
be the one using adaptive control techniques. The adaptive 
control techniques in robotics can be applied as a form of the 
well-known Model-Referenced Adaptive Control (MRAC) or 
Self-Tuning method [17), with the possibility of adaptation 
in feedforward or feedback loops. All such methods usually 
comprise the on-line schemes with recursive least-squares 
optimization criterion and short-term learning in which the 
past experience is completely forgotten. 

To summarize: Conventional adaptive control techniques 
robotics are efficient in the case of compensation of structured 
uncertainties, but in the presence of sensor data overload, 
heuristic sensor information, limits on real-time applicability 
and wide interval of unstructured uncertainties, the appli- 
cation of adaptive control is not sufficient for high-quality 
performance. 

Therefore, a solution to the robot control problem will prob- 
ably require combininig conventional approaches with new 
learning approaches in order to achieve good performance. 
For the robot control problem and learning we can identify 
three main paradigms: 

a) Iterative-analytical methods [18], [19]; 

b) Tabular methods [20]; 

c) Connectionist methods [3-15]. 

Iterative-analytical methods are based on successive at- 
tempts at following the same trajectory when control input 
values for each time instant in the trajectory are adjusted 
iteratively on the basis of the observed trajectory errors at 
similar time instants during previous attempts. These iterative 
learning algorithms may be very precise with fast convergence 
properties. But, on the other hand, drawback of such control 
techniques is that they are only applicable to the operations 
which are repetitive. Also, these algorithms have no capability 
of generalization on quite different movements. 

Tabular methods use the associative content — addressable 
memories, when the robot models are learned by storing ex- 
perience about command signals and current state coordinates 
in a memory. Each time a particular set of robot positions, 
velocities and accelerations is requested, the entire memory 



has to be searched for the closest experience. In this approach, 
the problems are great search time due great number of stored 
experience, the ways to measure similarity, and the methods 
of efficient generalization. 

Connectionist methods with distributed processing provide 
the implementation tools for complex input/output relations 
of robot's dynamics and kinematics. One of the main goals 
of dynamic learning methods is to find solution for the robot 
inverse dynamic problem. Let us explain the inverse dynamic 
problem of robot control in a computational framework. There 
are causal relation between robot driving torque and the re- 
sulting robot movement coordinates. Let P(t) denote the time 
history of driving torque and q(t) denote the time history of the 
robot internal coordinates during the trajectory. Also we can 
denote the causal relation between P and q using the functional 
F, i.e. F(P(-)) — q(-). If we want the robot to tracks desired 
trajectory gj, the problem to generate a desired driving torque 
P d which realizes q d% is equivalent to finding an inverse 
mapping of the functional F. The connectionist approach 
may, in principle, solve the problem of variable-coupling 
complexity and state-dependency of robot dynamic model, 
because neural networks through the process of training can 
approximate input-output mappings. In this way connectionist 
structure as part of decentralized feedforward control law can 
compensate wide range of robot uncertainties. Also, learning 
by neural networks is based on excellent association and 
generalization properties. The fast computational capability of 
neural networks enables real-time applicability of robot control 
algorithms. 

However, there are some problems in applying of the 
connectionist approach in robot control First, there is no 
quarantee in the learning processs for convergence of error 
to a local minimum. Second, neural networks implement only 
an approximation of inverse mapping of functional F, so 
the accuracy and robustness of this approximation may be 
questionable. Third, there is no systematic way for determi- 
nating the optimal topology of connectionist structure (number 
of layers and neural units, choice of activation functions, 
convergence parameters, etc.). Also, there are problems in 
specifying the "good" teacher ("good" basic control algorithm) 
for supervised learning robot control and specifying the op- 
timal set of input patterns for efficient generalization. One 
of main problems involves acceleration of the learning rules 
because connectionist structure may require a long training 
period to converge. All these problems are open field for 
connectionist and robotic researchers. Hence, our intention 
in the following sections is to provide solution to some of 
the above problems to achieve highly efficient robot learning 
control. 

III. Connectionist robot Controller 
wtth Decomposed Structure 

It is important to notice that the application of connectionist 
solution for robot dynamic learning is not limited only to 
the task with holonoraic constraints (pick-and-place operations 
and sliding assembly). It is also applicable for the tasks with 
nonholonomic constraints (mobile robot navigation, grinding 
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Fig. 1. Decentralized control structure with connectionist feedback error-learning. 



and contour following, robot grading, dexterous manipulation 
or assembly with a robotic hand). There are a few interesting 
connectionist solutions in this area [21], [22]. 

However, the primary interest in this paper is the learning of 
inverse dynamic model of robot mechanism for the tasks with 
holonomic constraints, where exact robot dynamics is gener- 
ally unknown. Hence, the proposed neural network models can 
be regarded as examples of the autonomous driving torque 
generator (Fig. 1). This connectionist structure is commonly 
used as part of feedforward controller in decentralized control 
algorithm. In this case, the feedback controller serves as a 
robust controller with aim to achieve low errors and perform 
high-quality learning, because the feedforward controller alone 
is not sufficient for accurate tracking. 

Training and learning by proposed connectionist structure 
is accomplished exclusively in on-line regime by feedback- 
error or driving torque error learning method [11] (Fig. I). 
This method is exclusively on-line method for robot control, 
but this control structure provides an internal teacher so that 
the control scheme works in an unsupervised manner, because 
we have no external teacher in this case. The adjustment of 
the network weights during the real-time control by feedback- 
error learning is more convenient than other learning structures 
as generalized or specialized learning [5]. 

The topology of proposed connectionist structure for robot 
learning control is defined by a four-layer perceptron with 
symmetric sigmoid function as activation functions in both 
hidden layers. The network has input layer with 3n neurons 
and an output layer with n neurons. This number of neurons in 
appropriate layers is determined according to number of degree 
of freedom for standard robot configuration. The activation 
function for input and output layer is the identity function. 
The number of neurons in the hidden layers is determined by 
simulation experiments and experience (12n neurons in the 
first hidden layer, 6n neurons in the second hidden layer). The 



neural network with proposed topology of fixed nonrecurrent 
multilayer network generates necessary driving torques in 
robot joints as a nonlinear mapping of robot desired internal 
coordinates, velocities and accelerations: 

Pi=9(w? kz q dl q d ,t'id) i=h-->n. (6) 

where PitR n is joint driving torque generated by neural 
network; t/$— are adaptive weighting factors between neuron 
j in a-th layer and neuron k in 6-th layer, g — nonlinear 
mapping. According to the integral model of robotic systems, 
the decentralized control algorithm with learning has the form 



Ui = /i(9d, U, ft. P) - KPuSi - KDae 
i- l,...,n. 



i - Kin J Cidt 



(7) 

where fi is the nonlinear mapping which describes nature of 
robot actuator model. 

Training and learning of proposed connectionist structure 
can be accomplished using well-known back propagation 
algorithm [23], In the process of training we can use two 
type of output error for back propagation algorithm. The first 
type of error is feedback control signal 

(8) 



. f n. 



t = l,.. 

where t^tBJ 1 is the output error for back propagation algo- 
rithm. 

But, in fact when we consider integral modelling of robot 
mechanism with robot actuators model, feedback control signal 
is not an output error for a neural network. Thus, we must 
to measure real driving torque or to calculate driving torque 
error signal 

= PI - Pi = a% + b% + c'ujfc i = l,...,n (9) 

where P[cR n is the real driving torque in the robot joints; 
a'efl n , 6VR n , c^ti?" are parameters of robot integral model 
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(proposed calculation of driving torque error is valid for 
commonly used robot DC-actuators). 

Although the proposed pure or naive neural network ap- 
proach without using knowledge about robot dynamics may 
be promising, it is important to notice that this approach 
will not be very practical because of high dimensionality 
of input-output spaces and long learning time. For example, 
with standard manipulation robots having 6 d.o.f., we have 
18 input variables and 6 output variables. Also, the number 
of trajectory patterns np (input and output variables) may be 
very large (10-100) if we want to examine the whole working 
space. Hence, for the robot training it would be necessary to 
present np 18 samples, i.e the training by pure connectionist 
models would require a neural network of impractical size 
and unreasonable number of repetition cycles. Therefore, we 
can conclude that the naive connectionist approaches are only 
applicable for the low-dimension robotic systems. 

Bassi. Bekey and Yeung [8], [9 J, [24] use the principle 
of functional decomposition to simplify the robot dynamics 
learning. This method includes a priori knowledge about 
robot dynamics which, instead of being particular knowledge 
corresponding to a certain type of robot models, incorporates 
the common knowledge of robot dynamics. In this way, the 
unknown input-output mapping is decomposed into simpler 
functions which are easier to learn because of the smaller 
domains. The process of decomposition has two stages [8]. 
The first simplification reduces three-vector dynamic function 
(external accelerations, internal positions and velocities) into 
the learning of several simpler two- vector basic subfunctions. 
The feedforward controller using standard back-propagation 
method can be constructed by combining the particular net- 
works according to their basic subfunctions. 

There are some problems in the application of this de- 
composition. First, the external and internal robot variables 
are mixed in basic dynamic equations and in this way the 
separation of variables may be very hard and artificial. Also, 
the learning domains are still large. Hence, in the second stage 
of decomposition, full decomposition is applied using a priori 
knowledge of basic subfunctions. The basic subfunctions for 
the fixed internal coordinates q can be composed from simpler 
mappings: constant, linear and quadratic terms. These simple 
subfunctions may be easily implemented in the feedforward 
controller using the context-sensitive approach [9]. 

The goal of our approach in synthesis of connectionist 
robot controller is to leave the control synthesis without a 
priori information about robot dynamics. Hence, it is very 
important to use all the available information regarding robot 
dynamic but only in general and specific form. The general 
knowledge for that purpose is conveniently incorporated into 
the structure of network. The way of attaining above goals is 
a decomposition of robot dynamics on simpler robot dynamic 
relations in the space of robot internal coordiantes. In this way, 
instead using a single neural network, training and learning 
is accomplished by several neural subnetworks which have 
simpler input-output relations that enable significant reduction 
of learning time. 

Two different methods of robot dynamics decomposition 
in the space of internal robot coordinates are proposed. We 



can see that in robot dynamics several terms can be identified 
which have a distinctive functional dependency. The first 
method is "3F-2SF" decomposition (decomposition of three- 
vector function (6) into two-vector subfunctions). Exactly, 
basic robot model (1) can be decomposed into two terms: 
first term H{q,B)q or Fi{q,q>B) 

second term or ftfaiii^) 

With this type of decomposition, instead of a multilayer 
perceptron with 3n input values (6), we have two multilayer 
perceptions with 2n inputs and n outputs for approximation 
of mapping F\ and F^: 

P^ 1 = F^»™™\q dy q d ) t = l,...,n. (10) 

I?™ = F 2 {w? k »™q dl qa) < = 1 n. (ID 

P^Pf^ + P?* 2 i=l,...,n. (12) 

where F\ is a nonlinear mapping for the first perceptron NN1; 
F2 is a nonlinear mapping for the second perceptron NN2; 
pf* m and p^ N2 are parts of robot dynamic model generated 
by perceptrons NNi and NN2; w& Nlab and ti$* 2a * are 
weighting factors for perceptrons NNI and NN2; Pi is 
driving torque at the output of connectionist structure. 

The topology of perceptrons NNI and NN2 are determined 
using similar activation functions and principles as in the 
previous case (input layer-2n neural units; first hidden layer- 
8n neural units; second hidden layer-4n neural units; output 
layer-n neural units). Training of both perceptrons is ac- 
complished synchronously by feedback-error learning method 
(Fig. 2). The feedback error signal or driving-torque error 
signal is transferred as output backpropagation error to the 
both perceptron outputs. 

The second method of decomposition includes deeper de- 
composition. It is "3F-JSF" decomposition (decomposition of 
three-vector function into three one-vector subfunctions). The 
robot model (2) can be decomposed into three terms: 
first term H(q, &)q or F x (q t q, 8) 
second term g(q,B) or F2(q,B) 
third term q T C(q> B)q or F t (q, q,B) I = 3, . . . , n + 2. 

We can see that each of these terms represents multipli- 
cations of an internal position-vector subfunction by robot 
internal variables or internal position subfunction itself, as is 
case for the third term. Our intention is to use multilayer 
perceptrons for learning the nonlinear position-vector sub- 
functions i/(g,0), C(q,B) and g{q,B). The total number of 
multilayer perceptrons in this case is n + 2 because position 
subfunctions C{q % B) has dimension n x n x n (to be precise, 
we have n subfunctions C(q,B) with dimension n x n). The 
multilayer perceptrons and the driving torque at the output 
of the connectionist structure are defined according to the 
following equations: 

learning of H(q, B) 

PX Kl =F l (w? k Nl * h t q d ) t = l,. ...n. J = l,...,n. 

(13) 

learning of g(q, B) 

P?* 2 = Ft{w? k NM 9 q d ) i = 1, . . . , n. (14) 
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Fig. 2. "3F-2SF" Decomposition connectionist structure wilh feedback error learning. 



learning of C l (q,Q) 

P;l m = Pi(™?k Wia \<I<i) i = l,...,n. m = l n. 

f = 3 n + 2. (15) 

tota/ owfpwf of connectionist feedforward structure 

Pi = £ + ^> V ' V, ' +2 </ + P^ 2 (16) 

where = 1, . . . ,n + 2) is a nonlinear mapping for 
perceptron NNi; P? Nt and P& A '' are the outputs of the 
perceptron TV Nl (i = 1, . . . , n; m— 1, . . . , «; I = 1, . . . , n + 
2;); uij^' 0 * are the weighting factors for perceptron /V/V/. 

In this case, number of neural units in network layers is 
defined according to specific features of robot dynamic models. 
For the first perceptron, which has a role to approximate 
H(q,9), we have the next topology: input layers neural 
units; first hidden layer-2n(n 4-1) neural units; second hidden 
layer— n(n + 1) neural units; output layer-u(n + l)/2 neural 
units. Number of neural units in the output layer is defined 
according to number of elements of the inertia matrix Hu(q, B) 
using symmetry properties of matrix H(q,8). The topology of 
perceptron NN2 which we are using for approximation of 
vector g(q t 8) is: input layer-n neural units; first hidden layer- 
4n neural units; second hidden layer-n neural units. Finally, 
the topology of n perceptrons for learning of subfunctions 
C l (q } 6) is: input layer-n neural units; first hidden layer- 
2n(n + 1) neural units; second hidden layer — ti(ti + 1) neural 
units; output layer-the number of neural units in this layer 
is different between these n perceptrons due properties of 
symmetry and antisymmetry. The whole number of outputs 
for all these n perceptrons is n(n — l)/3. 



Training of both perceptrons is accomplished synchronously 
using simlar principles as in previous cases by feedback-error 
learning method (Fig. 3). 

The decomposition steps of robot dynamics model with 
proposed neural network structure (number of neural unils in 
network layers) are illustrated on Fig. 4. The bias neural unit 
is included in all layers except for the output layer. In this 
particular case, number of the neurons in the output layers 
for "3F-ISF' decompositions is determined without using 
properties of symmetry and antisymmetry. 

It is important to notice that in both case of decomposition, 
outputs of connectionist structure defined by (12) and (16) are 
part of decentralized control algorithm according to (7). 

The main result of these decompositions is a significant 
reduction in the size of the trajectory input patterns. For 
example, we can perform learning analysis for standard robot 
configurations with 3-d.o.f. (degree of freedom) and 6 - d.o.f. 
by knowing the number of different patterns necessary to train 
the proposed neural network structures. The number of patterns 
is determined by number of samples np per each variable. In 
Table I are shown one example of learning analysis with the 
number of samples for 3 - d.o.f. and 6-d.o.f. manipulators 
using np = 10 samples per dimension. 

It is evident from Table I that the reduction in the number 
of patterns required for learning is 1 • 10 6 for n = 3 and 
1 * 10 12 for n = 6. On this way, breaking of a large 
space into smaller ones allows more practical learning the 
complex dynamics of high dimensional robotic systems. At the 
other side, instead of learning on single perceptron, we have 
simultaneous learning on several perceptrons. Hence, the real 
advantage of decomposed structure is a strong connection with 
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Fig. 3. "3F-1SF" Decomposition conncctionist structure with feedback error learning. 



TABLE I 

Example of Number of Patterns Needed for 3-d.o.f and 6-d.o.f Robots 



Method 


Number of 
samples 


n = 3 
np = 10 


n = 6 
np = 10 


pure 

connect, 
method 




to 9 


10 lft 


"3F-2SF" 
decomp. 
connect. 




2I0 6 


2 10 12 


"3F-ISF 
decomp. 
connect. 


np* 


If) 1 


10 6 



concurrent distributed processing nature of neural networks. 
With this goal, it is important to use on contemporary robot 
controllers a highly efficient parallel processing algorithms 
with real concurrent neural network hardware architectures 
[25H28J. 

We can conclude that when we use some aspect of robot 
dynamics information, principle of decomposition in the space 
of internal robot coordinates is a proper approach to real-time 
learning. 

iv. new connectionist feedforward control 
Algorithms With Fast Learning Properties 

The back propagation algorithm caused a significant break- 
through in the control application of multilayer preceplrons. 



The back propagation learning algorithm for layered neural 
nets performs a gradient search in the weight space, minimiz- 
ing cost function. One of the major drawbacks of this method 
is its slow convergence. Starting from a random initial state, 
the path to the global minimum is often strewn with local 
minimum, causing oscillations around ravines in the weight 
space. Sometimes the algorithm gets stuck in a local minimum 
and the training has to be repeated, starting from another initial 
state. 

There is a huge literature in the neural network field [16], 
[29] -[32] on how to speed up backpropagation. The one of 
common way to attaining this goal is by means of various 
methods from the field of numerical analysis. For example, it 
is very promising to use a modified version of error back prop- 
agation algorithm based on conjugate gradient descent method, 
because this method has quadratic convergence properties. 

In this paper, our intention is that instead acceleration of 
standard back propagation algorithm using standard methods 
from numerical analysis, consider the problem of adjusting 
the weights of internal hidden units as a problem of parameter 
estimation for linear deterministic systems by Recursive Least 
Square (RLS) method or as a filtering problem for linear 
stochastic systems by Kalman Filter (KF) approach [33]. Using 
these methods with time-varying learning rate yield to benefits 
for learning speed and generalization than are available with 
the standard back propagation algorithm. 



\$2 
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Kg. 4. Decomposition sieps of robot dynamics by connectionist structures. 

The another important feature for acceleration of learning 
algorithm is inclusion of adaptive gain in activation function 
for hidden layers [16]. The adaptive gain of a node in a con- 
nectionist network is a multiplicative constant that amplifies or 
attenuates the net input to the node. It is shown that using of 
gradient descent with respect to gain greatly increases learning 
speed by amplifying those directions in weight space that are 
successfully chosen by gradient descent on weights 1 16J. 

The proposed new algorithms are based on previously 
denned four-layer decomposed connectionist structures ("3F- 
2SF* and "3F-JSF") with appropriately defined parameters 
of network. 

The general forward network relations in process of training 
are described according to the next expressions: 

Forward Relations RLS & KF method 



s 2 (k) = W l \kfi l (k) ij(fc) = 1 
ol(k) = 1/(1 + Hxp(- fffl 2 (fc)^(fc))) - 0.5 

a=l t ...M ol{k)=l 
»*(k) = W 2 \k) T o 2 (k) 



(17) 

08) 
(19) 



ol(k) = 1/(1 + exp(-yUk)4(m ~ 0.5 

6=1, ...,L a ofck) = l 
s*(k) = W M {kfo*{k) 
y c (k) = st(k) = P™»(k) or P»»*{k) 



(20) 
(21) 



c = 1, . . . , n / = 1, . 



p=3 1 ...,n + 2(22) 



where s 2 (fc), s 3 (fc), s 4 (k)— are the output vectors for lin- 
ear parts of layers; o' 2 {k), o 3 (k)— are the output vectors of 
the hidden layers; gl{k), </?(/?)— are the adaptive gains of 
activation functions for hidden layers; W 12 = [w^ xL1 ], 
= [t^i 3 1+ ixL 2 K ^ 34 = KUixJ « weighting 
factors of the layers; ^(fc) is the input of network (robot 
internal positions, velocities and accelerations — m = 3u + 1); 
y(k) is output of network. 

As first approach for estimating the weights at the hidden 
layers, RLS method for parameter estimation of linear systems 
is applied. The aim of estimation is to define optima! values 
for matrices W 12 , W™, W 3 * using models of linear systems 
according to equations (17), (19), and (21). In application of 
this method, problem of specification of values for desired 
outputs of linear subsystems is arose. This problem is solved 
by inverse mapping of values for desired outputs of nonlinear 
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sybsystems (18) and (20) |34J. As second important feature, 
updated values of weighting factors in current iteration are 
back propagated to the input layer. Beside the on-line learning, 
the proposed algorithm is valid for off-line learning of robot 
dynamics as well as for any other mapping problem. 

The basic equations which describe new learning rules based 
on RLS method are given according to the next formulas: 



Learning Rules RLS method 



4'(k) = yi(k) = PZ(k) c=l,...,n (23) 



c=l,...,n j = l,....Li + l 



(24) 



p3(*)=l{p 3 (fc-D- 



P 3 (k-l)o 3 {k)o 3 (k) T P 3 (k-\) 
A 3 + o 3 (Jfc) I 'P 3 (fc- 



P 3 (fc-1) \ 
!)«»<*) J 



W M [k) = W"(k - 1) + P 3 (fc)o 3 (fc)[s 4 ' l (fe) 
-W 3 *(k-l) T o i (k)} T 

A2(*) = *?<*) 

j 

r;= l....,n j = l,... i L 2 + 1 
o 2d (k) = o z (k) + W u (k)6 4 (k) 
o^(k) = m^\ol d (k)\ 6=1,. ..,L 2 



(25) 
(26) 



(27) 
(28) 
(29) 



Initial conditions for weighting factors are generated by 
normal distribution with different random numbers: 

W 12 (0) = iV 12 (0, 1); W 23 (0) = N 23 ^ 1); 

iy 34 (0) = iV 34 (0 1 l); (39) 

Also, initial conditions for covariance matrices P are given in 
the following form: 

P l {o) = c 1 / mxm ; P 2 {0) = c 2 J Ll+ i xLl +i; 

F 3 (0) = c 3 / l , +1x l 3 +i; (40) 

where c 1 , c 2 , <? are positive numbers. 

The values of adaptive gains in activation functions are 
updated according to gradient descent on error that can be 
computed using standard back propagation algorithm: 

Learning Rules Adaptive Gains 

A^(fc) = y d c (k) - y e {k) = P!(k) - P c (k) = e?(fc) 

c=l,...,n (41) 
opl(k)=ol(k)(l-6l(k)) 6=1 (42) 



sl d (k) = In 



\ ^m>x / 

.0.5-(^) O ^(fc)_ 



P 2 (k) = 



W K (k) = W 23 (k - 1) + P 2 {k)o\k)[* M {k) 
-W 23 {k-lfo 2 (k)) T 
6Z(k) = s?(k)-"£w 23 (k)oj(k) 
j 

b = l,...,Li j = l + 1 

o M (Jt) = o 2 (k) + W 23 {k)6 3 (k) 
o 2 £ x (k) = mzx\o 2 a d (k)\ «=!,...,£, 



»?(*)=>>» 



/g 3 (k) 6 = 1,...,L 2 
(30) 

P 2 (fc-l) 0 2 (A:)o 2 (<:) T P 2 (fc-l) \ 
A 2 + t>WP2(k-l)o 2 (fc) J 
(31) 

(32) 

(33) 
(34) 
(35) 

M 

(36) 



0 - 5 + (^) o °"w 
°- 5 - («)*« 



/ 9 2 (fc) «=1,... 



pHk) - UpHk-i) - PHk-i)?m\kFr>(k-i)) 



lV 12 (fc) = IV 12 (* - 1) + P 1 (k)i 1 {k)[s 2d (k) 
-W l2 {k-\) r i\k)] T 

where A 1 , A 2 , A 3 — are the appropriate forgetting factors. 



(37) 
(38) 



3 b (k)=^Ai(k)w 3 ^k)yUk)op 3 b (k) 



9b 



"P. 



b = i,....M 
£(fc) = sf<fc -i) + <? 



(43) 



Ai(*)tu3f(*)Jpp?(*)*J(*) 

6=1. ..,L 2 (44) 
(k) = dl(k)(l-dl{k)) o=l,.. M Li (45) 

A2(*)i«2J(ifc)lop2(*)*J(fc) 



a = l...,Ll 



(46) 



where (J, Cg are learning rates for adaptive gains; oj)(fc) = 
og(As) + 0.5;oJ(*) = c£(*)+0.5. 

Using adaptive gains and time-varying learning rate in RLS 
method enables better accommodation of learning process to 
the robot dynamics. 

The KF approach for linear systems [33] is a second 
approach applied to estimating weighting factors in the net- 
work layers. In references [35], [36] similar strategy but for 
nonlinear stochastic problems using Extended Kalman Filter 
approach was considered with application for the classical 
XOR problem. In our, robotic case, the KF approach is related 
to linear state estimation ("filtering") problems where the 
following observation equations are observed : 

(47) 
(48) 



* i (fc) = ^<(*-l)oiW + »?(*) 

i 



(49) 



where {£(fc),f/(A;)} 318 mutually independent, zero-mean white 
gaussian noises with variances Ri and R 2 . It is important to 
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notice that these noises can be considered as pseudo-noises for 
tuning the gain in a Kalman filter. 

The learning algorithm via the KF approach by using a new 
specification for the desired states and errors in hidden layers 
is defined according to the following expressions: 

Learning Rules KF method 



s?(k) = wf(t) = P c r (fc) c=l,...,n (50) 

= «fw - fc(*) = P cW - E«#(*H(*) 

i 

c=l,...,n j=i,...,L 2 + l (51) 
K 3 (k) = P*(k - \)o 3 (k)[B% + o 3 (k) T P 3 (k - l)o 3 (fc)l-' 

(52) 

/>*(*) = [/ - K 3 (k)o 3 {k) T ]P 3 (k - 1) + R*I (53) 
W^ik) = W M {k-l) + K 3 (k)l* M (k)-W 3 \k-l) T o 3 (k)f 

(54) 

^(*) = -?•(*) -E 



c= l,...,n j = 1.....L2 + 
o 3 ^*) = „ 3 (jt) +• W 34 (lfc)A 4 (fc) 

= max ^(k)! fc = !,...,£, 



(55) 
(56) 
(57) 



/9 3 (fe) 6 = 1,...,L 2 



(58) 



K\k) = 


P 2 (k - 1)o 2 (Jk)[R| + o 2 (k) T P 2 (k - 












(59) 


P\k) = 


V- 


- K 2 (k)o 2 (k) T ]P 2 (k- 


-l) + R\l 


(60) 


W 23 (k) = 


W 23 (k-l)+K 2 (k)[s 3d (k)-W 23 (k- 


-1) T «W 










(61) 


%{k) = 


*?•(*) -E w "(W> 








b= \,...,Ld = + l (62) 




o^AO + W 23 ^)^*) 




(63) 


ofLW = 


nia*|nF(fc)| «=!,. 




(64) 


^(*) = 


In 


'0.5+ (^)^(*)" 


/<?«(*) « 


= l,...,Li 










(65) 


*'(*) = 


P 1 ^ - l)i\k)[Ri + i l {k) T P l {k - 




/»>(*) = 


[f- 


- K\k)i x {k) T ]P\k- 


- 1) + aJ/ 


(67) 



W 12 (k) = W i2 (k-\)+K x {k)[s 2d (k)-W u {k- l) T i l {k)) T 

(68) 

Initial conditions for weighting factors are generated by normal 
distribution with different random numbers or using previously 
"learned values" of weighting factors in off-tine learning: 

w l2 (o) = yv 12 (o, i); w 23 (o) = yv 12 (o, 1); 

W M (0) =N 34 (0,1); (69) 





Fig. 5. Industrial robot MANUTEC r3. 



Also, initial conditions for covariance matrices P and vari- 
ances R\ and R2 are given in following form : 

P l (0) = c'txmi = C 2 I Ll +i xLl+ l\ 

P 3 (0) = f; 3 /x, 2+1X i^+i; 

(70) 

/2j = cl; M=4; t= 1,2,3 (71) 



where c 1 



, c 2 , c 3 , 4 



4, 4*0 = 1 i 2 » 3 ) 303 positive numbers. 
The values of adaptive gains in activation functions are 
updated on same way according to previously defined equa- 
tions (41)-(46). Also, we can see that KF algorithm (50H68) 
is equivalent lo RLS algorithm (23H38) for R\ = 0. and 
Ri = 1. 

V. Simulation Examples 

In this section, simulation trials were run to verily the 
proposed connectionist algorithms in compensating the system 
uncertainties. The manipulation robot used for the simulation 
was a 6 d.o.f. industrial robot MANUTEC r3 [37] (Fig. 5). The 
parameters of the robot mechanism and actuators are listed in 
Tables 11 and in. 

In the learning phase, robot training is accom- 
plished using movement from point A with internal 
coordinates {gc(0.; -1.57;0.; 0.; 0.; 0.)} to point B 
{qe(0.7; -0.785; 0.5; 0.3; 0.3; 0.3)}. Time duration of 
movement is t = 1-s with a trapezoidal velocity profile. 
The PD feedback control was choosen with next values for 
local feedback gains: if Pe(62.04; C3.10; 25.74; 32.22; 27.89; 
30.31); A\Dc(2.88;2.99; 1.5; 1.24; 1.21; 1.05). We know 
that exact measuring of the link inertia and position of mass 
centre are very difficult. Hence in simulation experiments, the 
model uncertainties are defined by parametric disturbances 
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TABLE 0 

Parameters of Manutcc R3 Manipulation Mechanism 



7.5 10,0. 



Mechanism 
link no. 


1 


2 


3 


4 


5 


6 


Mass ft j 


50. 


56.5 


26.4 


28.7 


5.2 


0.01 


Length [m\ 


0.67 


0.5 


0.73 


0.01 


0.01 


0.01 


Moment of 


0. 


2.58 


0.279 


1.67 


1.25 


0. 


inertia Afts*" 2 ! 
Moment of 


0. 


2.73 


0.413 


1.67 


1.53 


0. 


inertia J y [kgm 2 ) 
Moment of 


1.16 


0.64 


0.245 


0.81 


0.81 


0.001 


inertia J t [kgm 2 \ 
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WITHOUT DECOMPOSITION 



TABLE UI 

Parameters of D.C. Actuators tor Robot MANUTEC R3 




4.5x10' 



3.0x io r 



'.5xl0 7 



?5 5.0 7.5 10.0 

LEARrma epochs 



Actuator 
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5 


6 


Ftg. 6. 
sitiun. 


Conv 


Mechanical 


126. 


252. 


72. 


24.8 


21.4 


8.6 




constant [Nm/A] 


















Electrical 


2.5 


5. 


1.428 


2.2 


1.74 


2.027 






constant \Vs/rad\ 
















5>10 T 


M. inertia 


0.0013 


0.0013 


0.0013 


0.00016 0.000 18 0.00004 




of rotor [kgm 1 ] 


















Viscous 


1132.979 


1091. 


411.6 


108. 


112.6 


3.6 




4x10 T 


coefficient [Nms/rad] 


















Rotor 


1. 


1. 


1. 


1. 


1. 








resistance [SI] 
















3x10 7 


Voltage 


7.5 


7.5 


7.5 


7.5 


7.5 


7.5 


I 




saturation [v] 














2x10 T 


Reduction 


105. 


210. 


60. 


99. 


79.2 


99. 






ratio 


















Reducer 


1. 


1. 


1. 


1, 


I. 


1. 




1x10 7 


efficiency 


















Rotor 


0.001 


0.001 


0.001 


0.001 


0.001 


0.001 




0 


induce vity [H] 



















5.0 



RLS ALGORITHM 

BP ALGORITHM 



4x10' 
3x10 7 



\.:. : 



IxlO 7 



with approximately 20% variation from nominal values for 
link mass and moment of inertia). 

In simulation experiments we were applied pure connec- 
tionist and decomposed "3F-2SF' network structure . The 
pure connection! st structure has next topology : input layer- 
19 neural units with identity activation function; first hid- 
den layer- 73 neural units with symetric sigmoid activation 
function; second hidden layer-37 neural units with symetric 
sigmoid activation function; output layer- 13 neural units with 
identity activation function. The decomposed structure has 
a next topology for each of multilayer perceptron: input 
layer-G neural units with identity function; first hidden layer- 
49 neural units with symmetric sigmoid activation function; 
second hidden layer-23 neural units with symmetric sigmoid 
activation function; output layer-6 neural units. 

It is important to notice that if we use directly in on- 
line feedback learning randomly initialized set of values for 
weighting factors, learning may be in some cases very slowly 
and impractical. Hence, our intention in this case is to use 
off-line learning with available a priori known informations 
about system model without model uncertainties. In process 
of off-line learning, set of weighting factors are generated 
which represents closed neighborhood of real optimal value for 
weighting factors. This set represents starting set for learning 
of real weighting factors in on-line feedback learning. In 
simulation off-line experiments, we use all previously defined 
connectionist structures for generating weighting factors that 



2.5 5 0 7.5 

LEABND1Q EPOCHS 



100 



Fig. 7. Total square error during training epochs for back propagation 
algorithm and RLS method without adaptive gains. 



are randomly initialized by Gaussian numbers with zero mean 
and variance o = 0.5. This idea is similar to learning in 
reference [5] where combined learning methods (generalized 
and specialized learning method) are used. 

The learning of robot dynamics is accomplished through a 
trial- and-error approach, when in successive epochs of training 
we present the same trajectory patterns. 

In simulation experiment with backpropagation algorithm, 
the learning rate for all layers is 77 = 0.01 while the momentum 
factor is a = 0.9. In the case of RLS connectionist algorithm, 
covariance matrices have a next initial values: 

P 1 (0) = L P 2 (0) = 1. P S (0) = 1. (72) 

while learning rates for adaptive gains have next values 



c; = o.oi c ff 3 = o.oi 



(73) 



In the case of KF approach, values of variance R\ and 
are defined according to next values R\ - 0.01 and R^ = 1. 
(t = 1,2,3), while covariance matrices have a next initial 
values: 

F 1 (0) = L P 2 (0) = 1. P 3 (0) = 1. (74) 

When comparing the pure connectionist structure and the 
"3F-JSF 1 decomposed connectionist structure, simulations ex- 
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Fig. 9. Position error for first d.o.f.-BP algorithm. 

periments with standard back propagation (BP) algorithm 
under the same conditions are performed. Fig. 6 shows the 
convergence results (i.e., total epoch square error during time 
with sampling period i - 0.1 s) for both connectionist 
structure. The result shows better learning of dynamic robot 
model and significant reduction of learning time for decom- 
posed connectionist structure. The convergence results of back 
propagation algorithm and RLS algorithm (Fig. 7) in the case 
of on-line learning but without adaptive gains are compared. 
Also, on the Fig. 8, total epoch square error in the case of 
RLS method with and without adaptive gains is presented. 
We can conclude from results, that especially with new RLS 
connectionist method and adaptive gains we can achieve the 
fast learning of robot dynamics. 

At Figs. 9 and 10, position error is shown for the first d.o.f. 
in the case of trajectory tracking with BP algorithm and RLS 
method (both with adaptive gains). The figure shows the first 
and 10th training trial. At Figs. 1 1 and 12, comparison of BP 
method, RLS method and KF method in on-line learning is 
given. In all cases, position error fir d.o.f. for 10-th epoch of 
training is shown. 

The figures show that with repetitive trials tracking errors 
using RLS method are considerably decreased and on this 
way highly efficient robot dynamic learning is accomplished. 
Also, with proposed KF method, good results are achieved 
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Fig. 11. Comparison of learning algorithms (BP-RLS) for first d.o.f. 
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Fig. 12. Comparison of learning algorithms (BP-KF) for first d.o.f. 

but it is important to notice that performance of this 
method extremely depends from initial values of variances 
R[ and B^. With aim to verify generalization properties 
of proposed algorithms, some simulation experiments are 
performed. In the generalization phase robot moves along 
a different trajectory from that used in the learning phase, 
exactly from point A qe(0.Z6; 0.95; 0.75; 0.1; 0.5; 0.) to point 
Bqe(l.2$\ 0.42; 0.22; 0.; 0.95; 0.1) with triangular velocity 
profile. 
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Fig. 13. Position error for the first d.o.f. in generalization phase. 




TIME (S) 

Fig. 14. Position error for the third d.o.f in generalization phase. 

The generalization results (position errors with learning 
using RLS method and without learning for the first and third 
joint) are presented in Figs. 13 and 14. 

We can see that with "learned" inverse dynamic robot 
model, tracking for quite different robot trajectories is quite 
satisfactory. 

VI. Conclusion 

The results presented indicate the feasibility of using highly 
efficient connectionist approaches for learning complex input- 
output relations of robot's dynamics. However, naive straight- 
forward neural network is not suitable for practical solutions 
in real-time, because of the increased dimensionality of input 
and output spaces and the long learning time. Hence, based on 
this facts proposed new connectionist control algorithms with 
decomposed "3F-2SF' AND "3F-1SF' structure use available 
information about robot dynamic but only in general and 
specific form. The use of decomposition principle in the space 
of internal robot coordinates enables the synchronous training 
of several multilayer perceptrons on the simpler input-output 
relations with significant reduction of learning time. 

We have studied acceleration of the learning algorithms 
for multilayer perceptrons in robot control and proposed a 
new algorithms that uses Recursive Least Square method and 



Kalman Filter approach for estimating of weighting factors 
in network layers. These algorithms have the feature that 
the learning rate is time dependent, which enables that with 
sufficiently convergent solution they assures faster learning 
than the generalized delta rule in standard backpropagation 
algorithms. Also, as result, adaptive capability of the connec- 
tionist controller to the system uncertainties was clarified. 

The proposed algorithms for connectionist robot learning 
are strictly related for the neural networks as static nonlinear 
input-output maps. For future research, it is very attractive 
idea of the using reccurent neural networks as well as process 
of evolution in the static multi-layer perceptrons. The neural 
networks that have recurrent nature, have potentially more 
computational power than pure feedforward neural networks. 
This type of neural networks are very promising for the future 
exploitation in robotic system, because these networks can 
solve some additional problems of system stability. At the 
other side, change in the neural network structure (propagation 
of neurons and degeneration of synapses) during the training 
process enable dynamic adaptation of the network capacity 
to the complex problem of the network interaction with the 
environment. 

The proposed neuromorphic structure with decomposition 
principle and fast convergence properties has a great possibility 
in real-time control and learning of manipulation robots. 
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